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Overview 

All data files acquired by the division are 
processed according to internal procedures. 
These procedures were developed several 
years ago and were geared specifically to nu- 
meric files or survey data. The increased use 
of database management systems and the in- 
crease in the acquisition of textual and car- 
tographic data have had an impact on 
particular steps in the procedures. Specific 
processing procedures for varying types of 
data are being investigated at the present 
time. To date it has been found that the gen- 
eral principles outlined for survey files can be 
followed for other types of data. 

The processing of data files refers to the 
verification of the data with the record layout 
and the preparation of the documentation or 
codebook. The latter was described in the 
previous issue of the Bulletin (Vol. 1-No. 3). 
Processing of the data file ensures that re- 
searchers can use the files without difficulty. 
Discrepancies and errors in the files are un- 
covered and documented. The MRA does not 
“clean” data in the same way as do many 
university-based archives. Errors, inconsis- 
tencies, and unidentified codes are docu- 
mented in the codebook, but are not cor- 
rected. As many of the data files are acquired 
from federal government departments and 
agencies, they must document the work un- 
dertaken by the department. Any “cleaning” 
of the data by the Archives would constitute a 
change to the original record and would no 
longer reflect the operations of the depart- 
ment or agency. 


Processing Steps 


The steps involved in the processing of the 
data files always appear to be straightfor- 
ward. It is only when the archivist tackles a 
difficult file that the complications arise. The 
first step is the preparation of two working 
copies of the file. These are made using 
9 track, 6250 BPI magnetic tapes with 
EBCDIC coding and IBM standard labels. A 
printout of the successful copy program is 
retained. If difficulties are encountered in 
completing the copying of the tape, it is use- 
ful to have a label dump. The information 
contained in the label will indicate any discre- 
pancies in the dataset name, label, etc. A 
partial listing or dump of the data is obtained. 
This usually includes the first and last five 
blocks of the data. A quick check at this point 
will sometimes show if there is anything un- 
usual about the data. The record layout can 
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be specifically checked by comparing the 
dump with the documentation. Major varia- 
tions between the logical record and the re- 
cord layout can be identified. The dump is 
usually done in character format; however, if 
the data show added characters or a lack of 
pattern, a dump in both character and hex- 
idecimal format may reveal the presence of 
data in non-character format such as packed 
decimal. Files for which these steps have 
been completed are considered to be pro- 
cessed to level one. 

For almost all files more detailed checks are 
undertaken. The data are examined both for 
the purpose of checking the adequacy and 
accuracy of the documentation provided by 
the donor, and for the purpose of finding 
errors, oddities, or other anomalies. More 
thorough data checking is undertaken by 
using statistical packages such as SAS or 
SPSS. Discrete variables are normally 
checked by means of frequency counts. Out- 
of-range codes are identified and investi- 
gated. The investigation may require con- 
tacting the donor to obtain more information 
on the codes in question. Codes that cannot 
be explained are indicated as errors. No data 
are corrected, changed, or deleted. 

Inter-field checks are done on variables 
having logical relationships with the values of 
other variables. For example, one field may 
state that a respondent is male while another 
contains answers to a question asked only of 
females. The second field should therefore 
contain a code for “non-applicable” or some 
such value. Inter-record checks are also un- 
dertaken when there is a logical relationship 
between the records. Once the data and re- 
cord checks are completed, the documenta- 
tion package is prepared. Files for which data 
checks have been completed are processed to 
level two. Two new copies of the data file are 
made on archival quality tapes and are then 
transferred for storage at two separate sites. 

The above steps are oriented to survey- 
type files. As mentioned earlier, the general 
principles hold for other types of data. The 
division has investigated the acquisition and 
processing of data from database manage- 
ment systems, in particular System 2000. It 
has been possible to produce a sequentially- 
unloaded character version of the database. 
Further investigation is required in this area. 
Procedures for the processing of textual data 
are under development as the division is ac- 
quiring more automated textual records. Car- 
tographic data are also being investigated 
and preliminary results indicate that many of 
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the procedural steps may be similar; 
however, the verification of the information 
will require access to additional peripheral 
hardware and software packages. 

As creators of data files have access to bet- 
ter software packages and are more con- 
cerned with the archival aspects at the time of 
survey design and file creation, many of the 
difficulties that presently occur will no longer 
exist. However, advancing technology and 
wider use of the computer in all fields will 
continue to provide archives with new prob- 
lems in ensuring the long-term use of data 
files. 


Te Irdering Machine 
_ Readable Data Files 


ite Police for data files held by the 
division originate from outside Ottawa. In 
order to reduce the time required to process 
requests for data, the following information 
is required from the originator of the request. 
The researcher should indicate the title of the 
file, the principal investigator or organization 
that created the data file, and the date(s) of 
the MRDEF. If data files being requested are 
updated yearly, the inclusive dates should be 
indicated. The researcher can contact the di- 
vision by telephone indicating the files re- 
quested, but all requests must be followed up 
in writing. The majority of the division’s 
holdings are on 9 track tape, at 6250 BPI, with 
standard IBM labels. The archivist receiving 
the request will verify that the format is ap- 
propriate. If the user requires an unlabelled 
tape or a different density, this should be 
specified. The division will try to accommo- 
date any special requirements. Requests for 
copies of data files will be processed as 
quickly as possible. The minimum time is 
usually three days as the service bureau used 
by the division is located near Toronto. A 
maximum limit of two weeks is allowed. The 
researcher will receive a copy of the data and 
the documentation, along with a form indica- 
ting the file specifications. An invoice will be 
sent to the researcher by Financial Services, 
Public Archives Canada, and payment 
should be made to the Receiver General for 
Canada. The cost of the service is indicated in 
the fee schedule outlined on page 2. It should 
be pointed out that these costs are always 
subject to change. 


Cont'd on page 2 
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“Ordering Machine Readable. . .” 


cont'd from page 1 


Machine Readable Archives Division Fee 
Schedule (effective April 1983) 


The schedule lists the costs charged to the 
user when requesting one of the services 
provided by the division. 


(1) Copy of Codebook: A charge of 10 cents per 
page (if only codebook is requested). 


(2) Tape Copying: The charge for tape copies is 
according to the following formula: 
20x + 20y where x = number of magnetic 
tapes 
y = number of files. 


For one file on one reel of tape the charge 
is: 20x + 20y or $20 + $20 = $40. 


(3) Data Extraction: Extracts of data from large 
files are provided and costs are based on 
this formula: 20x + 20y + (z-7) 
where x = number of magnetic tapes 

y = number of files 
z = person/hours greater than 7. 


(4) Statistical Analysis: Statistical analysis of 
data is offered ona limited basis. The direct 
computer costs will be charged to the 
user. All requests for analysis must be 
submitted in writing, at which time an 
estimate of the cost will be prepared. 


Update on the 1984 
IASSIST Conference 


Plans for the 1984 [ASSIST conference to be 
held in Ottawa, from May 15 to 18, are being 
finalized. Workshops on Data Library Man- 
agement, Micro-Computers, and Complex 
Data will be held on May 15. The workshops 
are intended to provide practical experience 
to those involved in the creation, use, man- 
agement, and distribution of data. 

The conference per se will be on May 16, 17 
and 18. The overall conference theme is 


“Coming of Age in the Brave New World.” In 
addition, there are three sub-themes. In 
terms of structure, each of the sub-themes 
will begin with a plenary session. The ple- 
nary will then break into three concurrent 
sessions that will explore specific aspects of 
the sub-theme. The three sub-themes are: 
privacy and confidentiality; advancing tech- 
nology and its impact on all aspects of data 
collection, analysis, and distribution; and 
changes in the “Information Empire.” The 
latter sub-theme will involve the changing 
roles and responsibilities of data archives and 
libraries and of organizations that create 
and/or collect data, as well as new types of 
data collection, particularly from an interna- 
tional perspective. 

The Local Arrangements Committee has 
been very active in arranging activities. Three 
receptions will be held as well as tours of the 
city, Statistics Canada, and hopefully a “high 
tech” firm. The registration package should 
be distributed by mid-March. The conference 
location will be the Park Lane Hotel, although 
information on other hotels will be included 
in the registration package. The cost of the 
conference is as follows (all prices being in 
Canadian funds): 


For [ASSIST members: 
Conference and Workshops: $ 90 
Conference Only: $ 80 
Workshops Only: 5D 


For non-members: 

Conference and Workshops: 
Conference Only: $ 90 
Workshops Only: $ 65 


A one-day fee of $50 for members and $55 
for non-members will be charged. The fees 
cover the reception, an evening buffet, re- 
freshment breaks, and tours. 

For anyone involved in the creation, use, 
analysis, collection and distribution of ma- 
chine readable data, the conference provides 
a useful opportunity to discuss problems, ex- 
change ideas, and learn new ways to solve 
problems. More information can be obtained 
from the IASSIST Program Committee, c/o 


$115 


Machine Readable Archives Division, 
395 Wellington Street, Ottawa, Ontario, 
K1A 0ON3 — (613) 593-7772. 


The Resoantian of Public Data Users 
(APDU) held its annual conference in Wash- 
ington, D.C., November 3 and 4. APDU, an 
American association, was organized in 1976 
to encourage interaction among data users, 
producers, and distributors. One hundred 
and twenty attended the 8th Annual Con- 
ference. The sessions focused on the ac- 
tivities of the federal government in surveys 
and products; facilitating use of data; micro- 
computers; issues in data linking and match- 
ing; and the 1990 Census. Users of the census 
data had an opportunity to indicate which 
products they would like. The conference 
provided an occasion for users of public data 
to meet with producers of the data and out- 
line their concerns and their needs. Users 
were also made aware of current studies 
being conducted by federal agencies. The 
personal contact with representatives from 
the federal agencies involved in data produc- 
tion was of more value than all the letters and 
telephone calls that users have to make to 
obtain information. The need for a similar 
association in Canada was expressed by the 
three Canadians who attended the meeting. 
It was felt that this type of meeting of public 
data users and producers provided invalu- 
able information to both groups. In order to 
determine the support such an association 
would have in Canada contact Sue Gavrel, 
Documentation and Public Service Section, 
Machine Readable Archives Division, Public 
Archives of Canada, 395 Wellington Street, 
Ottawa, Ontario, K1A 0N3, (613) 593-7772 if 
you are interested. 
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